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Abstract. Familiarity with the television set as a principal news and entertainment 
medium has allowed for a smooth transition of television programmes into the 
language classroom (Mishan, 2005). However, in the technology-rich environment 
we live in today, access to television programmes in foreign languages is not 
restricted to class. A good understanding of the reception of such authentic material 
can help inform its appropriate use by learners in different contexts. This study 
explores the reception of a Japanese variety show that features same-language text 
on screen by language learners of Japanese and native Japanese speakers through 
the use of eye-tracking technology and questionnaires (Sikkema, 2017). This paper 
reports on a section of the findings from the eye-tracking data analysis. Following 
a brief description of the research project, this paper describes the distribution of 
visual attention between different stimuli in the video stimulus by taking a closer 
look at participants’ fixation count and total fixation duration. 


Keywords: Japanese language learning, visual attention, multimodality, eye- 
tracking. 


1. Introduction 


Recent studies in Synchronous Computer-Mediated Communication (SCMC) have 
recognised the benefits of using eye-tracking technology to study online language 
learning behaviour, albeit in combination with qualitative methods and as part of 
Mixed Methods Research (MMR) designs (O’Rourke, 2012; Stickler & Shi, 2017; 
Stickler, Smith, & Shi, 2016). Such a combination of methods not only facilitates 
the triangulation of data sets; it also allows for studies to incorporate participants’ 
own perspectives into the interpretation of the data. 
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This study employs an MMR design to explore online viewing behaviour of Japanese 
Language Learners (JLLs) during an experimental task in which participants watch 
an excerpt of a Japanese variety show that features same-language text on screen 
(Sikkema, 2017). The main purpose of this research is to examine how JLLs at 
different stages of their language study receive such multimodal material to inform 
its appropriate use by learners. It implements eye-tracking technology in order to 
gain a better understanding of the distribution of subjects’ visual attention between 
stimuli that have the potential to contribute to learners’ understanding of the 
television programme. Questionnaires are used to gain insights into participants’ 
multimodal perception and attitudes towards using such audiovisuals as language 
learning material. 


This paper reports on a section of the findings from the eye-tracking data and 
discusses what types of stimuli participants visually attend to the most. Results on 
fixation count and total fixation duration are examined for four participant groups; 
three consist of JLLs while the fourth group comprises of native Japanese speakers as 
a point of reference for the analysis of learners’ behaviour (Sikkema, 2017, p. 291). 


This research is carried out as part of a larger project run at Dublin City University 
(DCU) that focusses on the reception of the type of same-language text on screen 
featured in Japanese variety shows (Sikkema, 2017, p. 290). 


2. Method 


Data has been analysed from 43 JLLs and five native Japanese speakers who took 
part in this research on a voluntary basis. For the purposes of this study, the JLLs 
were organised into three groups based on whether they had been on exchange to 
Japan at the time of data collection. Seventeen learners who had never been on 
exchange to Japan formed Group 1: pre-exchange. Group 2: exchange comprised 
of 20 participants who were on exchange in Japan for approximately three-and-a- 
half months at the time of data collection. The remaining six participants were in 
Group 3: post-exchange as they had spent a full year on exchange in Japan prior 
to their experiment sessions. The native Japanese speakers formed the Reference 
Group. The data sets consist of gaze data recorded with a Tobii X2-60 eye-tracker 
device, open and closed-ended responses collected through a pre-task and post-task 
questionnaire, and field notes taken during experiment sessions. 


The results have been generated with dynamic Areas Of Interest (AOIs). These 
were created in the video excerpt and aligned with particular stimuli on the screen 
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in order to extract gaze data associated with these stimuli. The AOIs have been 
defined with the help of data visualisation tools in the eye-tracking software which 
showed that faces of people appearing on the show and same-language text on 
screen that visually represent the dialogue gathered the most visual attention from 
all participant groups. The charts therefore display the total number and duration 
of fixations allocated to areas of the screen for faces, same-language text on screen 
that show speech, and other stimuli. The default settings of the Velocity-Threshold 
Identification (I-VT) filter? were used to generate these numbers. 


3. Results 


Figure | presents the total number of fixations that landed on faces, same- 
language text on screen, and other stimuli for each participant group. It illustrates 
that faces and same-language text on screen gathered the most visual attention 
for all groups. 


Figure 1. Total number of fixations (sum) per participant group 
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The proportions of the total number of times participants fixated on these particular 
stimuli were in respective group order: 80.6%, 78.3%, 82.8%, and 78.2%. When 


2. https://www.tobiipro.cony/siteassets/tobii-pro/learn-and-support/analyze/how-do-we-classify-eye-movements/tobii-pro-i-vt- 
fixation-filter.pdf 
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taking a closer look at the proportions for each of the two most attention-grabbing 
stimuli, it appeared that participants in Group | (pre-exchange) and the Reference 
Group looked more often at faces while Group 2 (exchange) and Group 3 (post- 
exchange) showed a tendency to look more at same-language text on screen. 
The proportions for the three learner participant groups seem to be more evenly 
distributed between faces and same-language text on screen while the Reference 
Group appeared to have a more even distribution between same-language text on 
screen and other stimuli. 


Figure 2 builds further on these findings and presents the total duration of all 
fixations combined for faces, same-language text on screen, and other stimuli for 
each participant group. It shows that the earlier identified attention-grabbers did 
not only gather the most visual attention, but were also fixated on the longest for 
all participant groups. 


Figure 2. Total fixation duration (sum) in seconds per participant group 
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The proportions of the total duration of fixations in seconds that participants spent 
looking at faces and same-language text on screen are in respective group order: 
81.7%, 80%, 81.7%, and 78.9%. As opposed to the previous chart, all participant 
groups spent most time looking at areas on the screen where faces of people appear 
on the variety show with the largest ratios for Group 1 (pre-exchange) and the 
Reference Group. 
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4. Discussion 


These results are in line with observations of Stickler and Shi (2017) in their 
study on online communication through a conferencing platform. Their findings 
showed that learners of the Chinese language looked most (70%) at content areas 
of the screen in which language teaching content is presented, followed with 20% 
of fixations directed towards social areas of the screen featuring the presence of 
participants (Stickler & Shi, 2017, p. 169). In addition, findings from the study 
of O’Hagan and Sasamoto (2016) on native Japanese speakers’ visual attention 
when watching a Japanese variety show demonstrated that faces attracted the most 
fixations from participants (pp. 47-50). This relates to the findings of this research, 
although it is suggested here that JLLs visually attend faces and same-language 
text on screen in a rather different way from native Japanese speakers. 


5. Conclusion 


This paper demonstrated that faces of people appearing on the show and same- 
language text on screen that represents the spoken dialogue gather the most 
visual attention from JLLs and native Japanese speakers in this study. Although 
participants in Group 2 (exchange) and Group 3 (post-exchange) look more often 
at same-language text on screen when compared to Group | (pre-exchange) and the 
Reference Group, the duration of all fixations together illustrate that faces of people 
on the show were fixated on the longest for all participant groups. These findings 
suggest that learners visually attend verbal stimuli more than native speakers and 
that social, non-verbal stimuli play a large role in participants’ viewing experience. 
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